Information processing apparatus and method, information processing system, and providing medium

ABSTRACT

The invention enables users to virtually attach information to situations in the real world, and also enables users to quickly and easily find out desired information. An IR sensor receives an IR signal transmitted from an IR beacon, and supplies the received signal to a sub-notebook PC. A CCD video camera takes in a visual ID from an object, and supplies the inputted visual ID to the sub-notebook PC. A user inputs, through a microphone, a voice to be attached to situations in the real world. The sub-notebook PC transmits position data, object data and voice data, which have been supplied to it, to a server through a communication unit. The transmitted data is received by the server via a wireless LAN. The server stores the received voice data in a database in correspondence to the position data and the object data.

CROSS REFERENCES TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 10/667,287, filed on Sep. 19, 2003, which is a continuation ofU.S. patent application Ser. No. 09/413,432, filed on Oct. 6, 1999,which claims priority to Japanese Patent Application No. P10-084622,filed in the Japanese Patent Office on Mar. 26, 1999, and JapanesePatent Application No. P10-296501, filed in the Japanese Patent Officeon Oct. 19, 1998, the entire contents of which are being incorporatedherein by reference.

This application relates to the commonly owned co-pending patentapplication “INFORMATION PROCESSING APPARATUS AND METHOD, INFORMATIONPROCESSING SYSTEM, AND PROVIDING MEDIUM,” Ser. No. 11/537,889, AttorneyDocket No. 3712174-00949.

BACKGROUND

The present invention relates to an information processing apparatus andmethod, an information processing system, and a providing medium. Moreparticularly, the present invention relates to an information processingapparatus and method, an information processing system, and a providingmedium, which are adapted to attach various virtual information to realworld environments or situations (such as a position and an object), andto present those various attached information.

Recently, systems and interfaces called Augmented Reality (abbreviatedas AR hereinafter) have been proposed. “Augmented Reality” is a genericterm for systems and interfaces based on the concept of providinginformation, which corresponds to situations in the real world such as aposition and an object, to users and offering a more comfortableinterface for the real world to the users.

FIG. 30 shows the concept of AR. As shown in FIG. 30, a user terminal,e.g., a computer that the user can wear on himself or herself (referredto as a wearable computer 122 hereinafter), is always held in anON-state to observe real world environments 121 and to presentappropriate information to the user as required.

A sensor 123 includes, e.g., an IR (infrared) sensor, a CCD (ChargeCoupled Device) video camera and a microphone to take in information(position information, ID information, voice information, etc.) of thereal world environments 121. The information taken in by the sensor 123is supplied to a server 125 via a network. The server 125 reads data,which corresponds to the supplied information of the real world, from aninternal database 125 a in which data corresponding to the informationof the real world is stored beforehand, and renders the read data to bedisplayed on an HUD (Head Up Display) 124 via a network. With such asystem, the user can acquire the data corresponding to the situations inthe real world environments 121.

One example of such an AR system is reported by Steven Feiner, BlairMaclntyre, and Doree Seligmann, “Knowledge-based augmented reality,”Communication of the ACM, Vol. 36, No. 7, pp. 52-62, August 1993. ThisAR system is a maintenance system for a laser beam printer. Assistinformation produced based on position information, which is obtained bya ultrasonic sensor, is indicated on a transmission type wearabledisplay mounted on the user's head.

Apart from the above-described AR system, there are known systemsallowing users to exchange various information about things in the realworld or various information in the virtual world via a network or thelike. For example, information exchange service systems called anon-line bulletin board system (BBS) and chat over the Internet have beenalready utilized. With one of personal computer (PC) communicationservices, a bulletin board (forum) or the like subdivided depending uponfavorable items of users is provided so that the users can exchangeinformation via the bulletin board. This form of service often plays apredominant role in the user community. The BBS is usually utilized insuch a manner that the user retrieves an interesting bulletin boardthrough a search service based on the name of the bulletin board, etc.,and then browses information written on the retrieved bulletin board orwrites new information on it.

However, the conventional AR system described above is problematic inthat the assist information displayed on the HUD or the like is limitedto the information previously registered in a database stored in acomputer, and such information (annotation) as popping into the user'shead during maintenance work cannot be newly added to the database.Specifically, the above-described AR system is only able to readinformation corresponding to situations in the real world from among theinformation previously stored in the database, and to present the readinformation to users. In other words, the conventional AR system hasbeen merely a context-aware browser.

When users receive the information exchange services such as the BBS andchat over the Internet or PC communication, the users face such aproblem that they have to retrieve, e.g., interesting bulletin boardsthrough a search service based on the name of each bulletin board, etc.However, a large number of bulletin boards, etc. exist on the Internet,and it is not always easy to quickly find out a desired one of thosebulletin boards, etc. on which desired information is written.

SUMMARY

In view of the state of art set forth above, an object of the presentinvention is to provide an information processing apparatus and method,an information processing system, and a providing medium, with whichinformation (such as image information, text information and voiceinformation) corresponding to situations (such as a position and anobject) in the real world can be presented to users, while users canattach, to situations in the real world, information corresponding tothe situations in the real world, and with which users can quickly andeasily find out desired information.

The above object of the present invention is achieved by an informationprocessing apparatus for communicating with another informationprocessing apparatus, wherein the apparatus includes a specificinformation input unit for taking in specific information to specifysituations in the real world; an attached information input unit fortaking in attached information to be attached to the situations in thereal world specified by the specific information that is taken in by thespecific information input unit; a transmitting unit for transmittingthe attached information and the specific information; a receiving unitfor receiving the attached information transmitted from the anotherinformation processing apparatus corresponding to the transmittedspecific information; and a presenting unit for presenting, asinformation related to the specified situations in the real world, theattached information having been received.

Also, the above object of the present invention is achieved by aninformation processing method implemented in an information processingapparatus for communicating with another information processingapparatus, wherein the method includes a specific information input stepof taking in specific information to specify situations in the realworld; an attached information taking-in step of taking in attachedinformation to be attached to the situations in the real world specifiedby the specific information; a transmitting step of transmitting theattached information and the specific information; a receiving step ofreceiving the attached information transmitted from the anotherinformation processing apparatus corresponding to the specificinformation transmitted in the transmitting step; and a presenting stepof presenting, as information related to the specified situations in thereal world, the attached information having been received.

Further, the above object of the present invention is achieved by aninformation processing apparatus for communicating with anotherinformation processing apparatus, wherein the apparatus includes areceiving unit for receiving specific information to specify situationsin the real world and attached information attached to the situations inthe real world specified by the specific information, the specificinformation and the attached information being both transmitted from theanother information processing apparatus; a storage unit for storing thereceived specific information and attached information; a reading unitfor reading the attached information corresponding to the specificinformation from the storage unit; and a transmitting unit fortransmitting the attached information having been read.

Still further, the above object of the present invention is achieved byan information processing method implemented in an informationprocessing apparatus for communicating with another informationprocessing apparatus, wherein the method includes a receiving step ofreceiving specific information to specify situations in the real worldand attached information attached to the situations in the real worldspecified by the specific information, the specific information and theattached information being both transmitted from the another informationprocessing apparatus; a storing step of storing the received specificinformation and attached information; a reading step of reading theattached information corresponding to the specific information stored inthe storing step; and a transmitting step of transmitting the attachedinformation having been read.

Still further, the above object of the present invention is achieved byan information processing system including at least a first informationprocessing apparatus including a specific information taking-in unit forinput specific information to specify situations in the real world, anattached information input unit for taking in attached information to beattached to the situations in the real world specified by the specificinformation, a transmitting unit for transmitting the attachedinformation and the specific information, a receiving unit for receivingthe attached information transmitted from a second informationprocessing apparatus corresponding to the transmitted specificinformation, and a presenting unit for presenting, as informationrelated to the specified situations in the real world, the attachedinformation having been received; and a second information processingapparatus including a receiving unit for receiving the specificinformation and the attached information both transmitted from the firstinformation processing apparatus, a storage unit for storing thereceived specific information and attached information, a reading unitfor reading the attached information corresponding to the specificinformation from the storage unit, and a transmitting unit fortransmitting the attached information read by the reading unit; thefirst information processing apparatus and the second informationprocessing apparatus being connected via a network.

Still further, the above object of the present invention is achieved bya providing medium for providing a computer-readable program to executea processes including the steps of a specific information input step oftaking in specific information to specify situations in the real world;an attached information input step of taking in attached information tobe attached to the situations in the real world specified by thespecific information; a transmitting step of transmitting the attachedinformation and the specific information; a receiving step of receivingthe attached information transmitted from the another informationprocessing apparatus corresponding to the transmitted specificinformation; and a presenting step of presenting, as information relatedto the specified situations in the real world, the attached informationhaving been received.

Still further, the above object of the present invention is achieved bya providing medium for providing a computer-readable program to executea process including the steps of a receiving step of receiving specificinformation to specify situations in the real world and attachedinformation attached to the situations in the real world specified bythe specific information, the specific information and the attachedinformation being both transmitted from the another informationprocessing apparatus; a storing step of storing the received specificinformation and attached information; a reading step of reading theattached information corresponding to the specific information stored inthe storing step; and a transmitting step of transmitting the attachedinformation having been read.

In the present invention, the specific information includes at least oneof image information of the real world, position information of the realworld, and object information of the real world. Also, the situation inthe real world is an object in the real world, the specific informationis an ID attached to the object in the real world, and the attachedinformation transmitted from the transmitting unit is information thatis related to the object in the real world and can be browsed andexchanged. Further, the situation in the real world is a position in thereal world, the specific information is position information of the realworld, and the attached information transmitted from the transmittingunit is information that is related to the position in the real worldand can be browsed and exchanged.

Additional features and advantages are described herein, and will beapparent from the following Detailed Description and the figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram showing a schematic entire configuration of aninformation processing system of the present invention.

FIG. 2 is an illustration for explaining the concept of Augment-ableReality according to a first embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration of one example of anAugment-able Reality system according to the first embodiment of thepresent invention.

FIGS. 4A and 4B are representations for explaining a visual ID.

FIG. 5 is a block circuit diagram showing one example of configurationof a sub-notebook PC in FIG. 2.

FIG. 6 is a representation showing one example of a screen image of anHUD.

FIGS. 7A to 7C are representations for explaining the operation ofmoving an icon between a personal information tray and a context-awarearea.

FIG. 8 is a representation for explaining a time machine mode.

FIG. 9 is a flowchart for explaining the process of producinginformation to be attached to situations in the real world andregistering the information in a database in the Augment-able Realitysystem.

FIG. 10 is a flowchart for explaining the process of reading anddisplaying the information, which has been attached to situations in thereal world, from the database.

FIG. 11 is a representation showing one example of a screen image ofJava applet in a computer (41) in FIG. 3.

FIG. 12 is a representation showing one example of E-mail transmittedfrom a computer (42) in FIG. 3.

FIG. 13 is a schematic view showing one example of conceptualconfiguration of a system according to a second embodiment of thepresent invention.

FIG. 14 is a representation showing one example of an initial screenimage on a client PC in a BBS search using an ID.

FIG. 15 is a representation showing one example of a screen image of abulletin board retrieved in accordance with the ID.

FIG. 16 is a representation showing one example of a screen image whenan ID is newly registered.

FIG. 17 is a flowchart showing a process flow when retrieval andbrowsing of and writing on a bulletin board corresponding to an articleID are performed, or a bulletin board is newly registered on the clientPC side in the system according to the second embodiment.

FIG. 18 is a flowchart showing a process flow when the bulletin boardcorresponding to the article ID or a request for new registration of abulletin board is transmitted to the client PC from the server side inthe system according to the second embodiment.

FIG. 19 is a flowchart showing a process flow on the server side when anumber of bulletin boards corresponding to the same ID are found by asearch.

FIG. 20 is a flowchart showing a process flow on the client PC side whena list of the names, abstracts, etc. of the number of bulletin boardsare transmitted from the server side.

FIG. 21 is a schematic view showing one example of conceptualconfiguration of a system according to a third embodiment of the presentinvention.

FIG. 22 is a representation showing one example of a display screenimage of a bulletin board retrieved using the specific name of somerestaurant as a search condition at the current position.

FIG. 23 is a representation showing one example of a display screenimage of a bulletin board retrieved from bulletin boards at respectivepositions near the current position in match with additional searchconditions.

FIG. 24 is a flowchart showing a process flow for new registration of abulletin board related to the current position.

FIG. 25 is a flowchart showing a process flow for, e.g., retrieval andbrowsing of and writing on a bulletin board corresponding to the currentposition.

FIG. 26 is a flowchart showing a process flow on the server side whennew registration of a bulletin board corresponding to the currentposition is requested from the client PC.

FIG. 27 is a flowchart showing a process flow for, e.g., retrieval andbrowsing of and writing on a bulletin board in relation to the currentposition.

FIG. 28 is a flowchart showing a similarity determining algorithm in amethod of using, as a “position” of convenience, a pair of base stationID and electric field intensity when position recognition is carried outusing the PHS.

FIG. 29 is a block circuit diagram showing one example of concreteconfiguration of principal parts of client PCs (304, 404) applicable tothe second and third embodiments.

FIG. 30 is an illustration for explaining the concept of conventionalAugmented Reality.

DETAILED DESCRIPTION

Preferred embodiments of the present invention will be described withreference to the drawings.

FIG. 1 shows an entire configuration of one working mode of the presentinvention in which an information processing apparatus and method, aninformation processing system, and a providing medium are embodiedaccording to the present invention.

Referring to FIG. 1, a first information processing apparatus (which isprimarily used by users and is referred to as a user terminal 200hereinafter) is shown, which includes a real-world-environmentinformation recognizing unit 201 which serves as specific informationinput unit means for taking in specific information to specifysituations (context information such as information regarding an objectimage, an object ID, a position and a voice) in the real worldenvironments 1; a real-world-environment information analyzing unit 202for analyzing the information taken in by the real-world-environmentinformation recognizing unit 201; an attached information input unit 203for taking in attached information to be attached to situations (such asan object and a position) in the real world; a communication unit 207for communicating information between the user terminal 200 and a secondinformation processing apparatus, for example, (which is primarilyinstalled on the service providing side and is referred to as a server210 hereinafter); and a control unit 204 for receiving desiredinformation from the server 210 based on the information analyzed by thereal-world-environment information analyzing unit 202, and for producinga video signal, a voice signal, etc., based on the desired information.The user terminal 200 further includes a display unit 206 for displayingthe video signal from the control unit 204 on a screen, and a voiceoutput unit 205 for outputting the voice signal from the control unit204. Additionally, the functions of the user terminal 200 in thisworking mode may be implemented in the form of hardware or software.

Furthermore, in this working mode of the present invention, the server210 includes an information accumulating unit 213 for storing thespecific information to specify situations in the real world and theattached information attached to the situations in the real world whichare specified by the specific information, both types of the informationbeing transmitted from the user terminal 200; a communication unit 211for receiving the specific information and the attached informationtransmitted from the user terminal 200, and for transmitting, to theuser terminal 200, the attached information read from the informationaccumulating unit 213 corresponding to the specific information; and acontrol unit 212 for controlling processes of storing and readinginformation in and from the information accumulating unit 213 andprocesses of transmitting and receiving data to and from thecommunication unit 211. Additionally, the functions of the server 210 inthis working mode may be implemented in the form of hardware orsoftware.

In this working mode of the present invention, the user terminal 200 andthe server 210 are connected to each other via a network, for example,to construct an information processing system (network system).

As a more concrete form, a first embodiment of the network shown in FIG.1 will be described below with reference to FIGS. 2 to 12.

An information processing apparatus (e.g., a user terminal) according tothe first embodiment of the present invention is an informationprocessing apparatus for communicating with another informationprocessing apparatus (e.g., a server), and mainly includes a CCD videocamera 21, shown in FIG. 3, as one example of an image input unit fortaking in an image of the real world; an IR sensor 22, shown in FIG. 3,as one example of a position detecting unit for detecting a position inthe real world; an image processing unit 61, shown in FIG. 5, as oneexample of an object identifying unit for identifying an object in thereal world; a microphone 24, shown in FIG. 3, as one example of anattached information input unit for taking in attached information,which is to be attached to the real world, corresponding to the positioninformation outputted from the position detecting unit or the objectinformation outputted from the object identifying unit; a step S7, shownin FIG. 9, as one example of transmitting unit for transmitting theattached information taken in by the attached information input unit,the position information and the object information; a step S29, shownin FIG. 10, as one example of a receiving unit for receiving, from theanother information processing apparatus, the attached informationtransmitted from the transmitting unit and corresponding to the positioninformation and the object information; and a step S31, shown in FIG.10, as one example of a display control unit for controlling display ofan image produced by combining the attached information received by thereceiving unit and the image information taken in by the image inputunit.

Also, an information processing apparatus (e.g., a server) according tothe first embodiment of the present invention includes a step S8, shownin FIG. 9, as one example of a receiving unit for receiving the attachedinformation to be attached to the real world, the position informationrepresenting a position in the real world, and the object informationfor identifying an object in the real world, these three types ofinformation being transmitted from another information processingapparatus (e.g., a user terminal); a database 12a, shown in FIG. 3, asone example of a storage unit for storing the attached informationreceived by the receiving unit in correspondence to the positioninformation and the object information; a step S27, shown in FIG. 10, asone example of a reading unit for reading, from the storage unit, theattached information corresponding to the position information and theobject information; and a step S28, shown in FIG. 10, as one example ofa transmitting unit for transmitting the attached information read bythe reading unit.

An AR system according to the first embodiment of the present inventiondiffers from the conventional AR system in that information flowsbidirectionally and is not merely “augmented.” In view of the abovemeaning, the present AR system is referred to as an Augment-able Realitysystem hereinafter. Also, in the following description, conventionaldesktop, laptop, and notebook computers are collectively referred to asnormal computers for the purpose of clear distinction from wearablecomputers. FIG. 2 illustrates the concept of Augment-able Realityaccording to the first embodiment of the present invention. In the firstembodiment, a wearable computer 2 that the user can wear on himself orherself is employed as one example of a user terminal. Of course, theuser terminal may be, e.g., a portable computer instead of a wearablecomputer. The wearable computer 2 is always held in an ON-state toobserve situations in the real world environments 1 and to presentappropriate information to the user as required.

A sensor 3 includes, e.g., an IR (infrared) sensor, a CCD video cameraand a microphone to take in context information (position information,ID information, voice information, etc.) of the real world environments1. The context information taken in by the sensor 3 is supplied to aserver 5 via a network. The server 5 reads data, which corresponds tothe supplied context information, from an internal database 5 a in whichdata corresponding to the context information is stored beforehand, andrenders the read data corresponding to the supplied context informationto be supplied to and displayed on an HUD 4 via a network. With such asystem, the user having the wearable computer 2 put on himself orherself can acquire the data corresponding to the situations in the realworld environments 1.

Also, the user having the wearable computer 2 put on himself or herselfcreates, with the aid of the sensor 3, image data, text data, voicedata, etc. which are to be attached (or affixed) to the situations inthe real world environments 1. The attached data thus created issupplied to the server 5 via the network. The server 5 stores thesupplied attached data in the database 5 a in correspondence to thecontext information of the real world environments 1. The server 5 alsorenders the attached data, which has been newly stored in the database,to be displayed on the HUD 4 via the network as required.

When another user having a wearable computer 8 put on himself or herselfis in the same situation as the user having the wearable computer 2 puton himself or herself, the server 5 presents via the network the datacorresponding to the situation to the other user having the wearablecomputer 8 as well.

The information corresponding to the situations in the real world canalso be attached and browsed from normal computer environments. Forexample, when the user having the wearable computer 2 attachesinformation to a certain room, the attached information can be accessedfrom a normal computer 6 through, e.g., a map on the Web. When anotheruser transmits an E-mail addressed to a “meeting room A” from a normalcomputer 7, the server 5 stores the received E-mail in the internaldatabase 5 a. Then, when the user having the wearable computer 2 comesto the “meeting room A,” the contents of the E-mail sent to the “meetingroom A” are given to that user.

A configuration of the Augment-able Reality system according to thefirst embodiment will be described below with reference to FIG. 3. It isassumed that the Augment-able Reality system shown in FIG. 3 isinstalled in a building.

A wearable unit 11 mainly includes the CCD video camera 21, the IRsensor 22, a mouse 23, the microphone 24, an interface box 25, asub-notebook PC (Personal Computer) 26, an HUD 27, and an earphone 28which are all carried with the user. The wearable unit 11 (thesub-notebook PC 26) is connected to a server 12 via a wireless LAN(Local Area Network) 16. The server 12 includes the database 12atherein. A normal computer 41 is connected to the server 12 via a WWW(World Wide Web) interface 13. A normal computer 42 is connected to theserver 12 via an E-mail interface 14.

The Augment-able Reality system shown in FIG. 3 employs, e.g., both anIR beacon 17 and a visual ID for recognizing situations (such as aposition and an object) in the real world. The IR beacon 17 is installedin each room of the building, and transmits an IR signal at intervals ofpredetermined time. This IR signal enables a position recognition to bemade at a room level. Also, recognition at an object level can be madeby pasting a visual ID 52, which is formed by printing, e.g., atwo-dimensional bar code (so-called cybercode) shown in FIG. 4B, onto anobject such as a video cassette recorder (VCR) 51 shown in FIG. 4A.

The CCD video camera 21 takes in an image of the real world. The CCDvideo camera 21 also takes in two-dimensional bar code data of thevisual ID 52 pasted onto the object 51. The image data and thetwo-dimensional bar code data of the visual ID 52 thus taken in aresupplied to the sub-notebook PC 26 via a PCMCIA (Personal ComputerMemory Card International Association) interface 31.

The IR sensor 22 receives the IR signal transmitted from the IR beacon17, and then supplies the received signal to the sub-notebook PC 26 viathe interface box 25.

The user operates the mouse 23 to input information to the sub-notebookPC 26. Also, when the user inputs a voice through the microphone 24, theinputted voice data is supplied to the sub-notebook PC 26.

The sub-notebook PC 26 will be described in more detail with referenceto FIG. 5.

The image data and the two-dimensional bar code data taken in by the CCDvideo camera 21 are supplied to the image processing unit 61 via thePCMCIA interface 31. The image processing unit 61 executes apredetermined process on the supplied image data, and supplies theprocessed image data to a control unit 65. Further, the image processingunit 61 recognizes the supplied two-dimensional bar code data,identifies the object 51, and supplies the object data to the controlunit 65.

The signal received by the IR sensor 22 is supplied to a positioninformation detecting unit 62 via the interface box 25. The positioninformation detecting unit 62 detects the position of a room (identifiesa room) based on the supplied signal, and supplies the detected positiondata to the control unit 65.

The data inputted through the mouse 23 is supplied to the control unit65. The voice data inputted through the microphone 24 is supplied to avoice processing unit 63. The voice processing unit 63 executes apredetermined process on the supplied voice data, and supplies theprocessed voice data to the control unit 65.

The control unit 65 transmits the supplied position data, object data,image data and voice data to the server 12 via a communication unit 32.The transmitted data is received via the wireless LAN 16 by acommunication unit (not shown) incorporated into the server 12. Acombining unit 66 combines an output of the CCD video camera 21 and anoutput of the control unit 65 with each other, and supplies the combinedoutput to the HUD 27 for display on it.

The server 12 stores the received image data and voice data in thedatabase 12a in correspondence to the position information and theobject information.

The process of reading the information attached to the situations of thereal world in the first embodiment will be explained below.

When the position data and the object data are supplied respectivelyfrom the position information detecting unit 62 and the image processingunit 61, the control unit 65 transmits the position data and the objectdata to the server 12 through the communication unit 32. The transmittedposition data and object data are received via the wireless LAN 16 bythe communication unit incorporated in the server 12.

The server 12 reads, from the database 12a, the image data and the voicedata corresponding to the transmitted position data and object data. Theread image data and voice data are transmitted to the sub-notebook PC 26via the communication unit.

The transmitted image data and voice data are received by thecommunication unit 32 of the sub-notebook PC 26 via the wireless LAN 16.The communication unit 32 supplies the received image data and voicedata to the control unit 65. The control unit 65 creates iconscorresponding to the supplied image data and voice data, respectively,and supplies the created icons to the combining unit 66. The controlunit 65 also outputs the supplied voice data through the earphone 28.The combining unit 66 combines the corresponding icon with the suppliedimage data of the current sight, followed by displaying a combined imageon the HUD 27. As a result, the user having the wearable unit 11 put onhimself or herself can acquire information corresponding to thesituations of the real world.

One example of a display screen image on the HUD 27 will be describedbelow with reference to FIG. 6. The following example of a displayscreen image represents the case that the user operates a VCR (object)at a location of an editing studio and pastes, as a voice memo, a pointfound out by the user about the VCR (object).

As shown in FIG. 6, the display screen image on the HUD 27 is dividedinto a personal information tray 71 and a context-aware area 72. In thepersonal information tray 71, information carried with the user, such asa voice memo and an image snap shot, is indicated by a voice icon memo75.

The context-aware area 72 corresponds to the situation of the realworld, and the screen image in the area 72 varies each time thesituation is changed with movement of the user. The context-aware area72 is divided into a location field 73 corresponding to the positioninformation at a room level, and an object field 74 corresponding to therecognized object. When the user is present in the editing studio andstands in front of a VCR, an icon 76 representing that the user is nowpresent in the editing studio is indicated on the left side of thelocation field 73, and an icon 77 representing that the user is nowstanding in front of the VCR is indicated on the left side of the objectfield. In the location field 73, a voice memo that is informationattached to the editing studio is represented as an icon 78. By clickingthe icon 78 with the mouse 23, the user can listen the correspondingvoice through the earphone 28. In the object field 74, informationattached to the VCR is represented as a voice memo icon 79 and an imagesnap shot icon 80.

One example of screen images displayed when the user attachesinformation created in the personal information tray 71 to thecontext-aware area 72, will be described below with reference to FIGS.7A to 7C.

When the user clicks a RECORD button 84 with the mouse 23, a voice noteicon 91 is produced (FIG. 7A) and is displayed in the personalinformation tray 71 (FIG. 7B). Then, when the user operates the mouse 23to drag and drop the icon 91 in the location field 73 for the editingstudio, the icon 91 is pasted to the room of the editing studio (FIG.7C). The result of the above operation is transmitted to the server 12via the wireless LAN 16. The server 12 registers the received data inthe database 12a and provides the data to other users as required.

One example of a screen image in a time machine mode will be describedbelow with reference to FIG. 8. If not specified otherwise, thefollowing description is made in connection with the case that today isApril 2 and the user goes back one day, i.e., to April 1 and returns tothe front of the VCR in the meeting room.

An icon 101 representing that the user is in the meeting room on April 1is indicated on the left side of the location field 73. By clicking acontext switch button 107 on the left side of the icon 101, the user cango back to the rooms which the user has visited so far. By clicking acontext switch button 108 on the left side of the icon 102, the user cango back to the objects (things) which have been recorded so far.

A time indicator 109 indicates the date and time of the situationdisplayed in each field. In this case, it is seen from the timeindicator 109 that the meeting room at 12:15 one day before, i.e., onApril 1, is displayed in the location field 73, and the VCR at 12:15 onApril 1 is displayed in the object field 74. Further, it is seen fromthe time indicator in the personal information tray 71 that the currentdate and time is 12:15 on April 2. By clicking up and down buttons (U,D) in each field, the user can move to the situation in the future orpast. For example, by pushing an up button U73 in the time indicator 109displayed in the location field 73, the user can move to the meetingroom on April 3 (tomorrow), for example, and attach information to thatsituation.

Icons 103 to 106 represent items of information which are attached tothe respective fields.

The process of creating information to be attached to situations in thereal world and registering the information in the database 12 a, whichis prepared on the network, in the Augment-able Reality system shown inFIG. 3 will be described below with reference to a flowchart of FIG. 9.

First, when the user having the wearable unit 11 put on himself orherself moves to a predetermined room (e.g., the editing studio), the IRsensor 22 receives the IR signal transmitted from the IR beacon 17 andsupplies the received signal to the position information detecting unit62 via the interface box 25 in step S1.

In step S2, the position information detecting unit 62 detects, based onthe supplied signal, the position at a room level (e.g., the fact thatthe room is the editing studio), and supplies the detected position datato the control unit 65.

In step S3, the CCD video camera 21 takes in the visual ID (e.g.,two-dimensional bar code) 52 pasted onto the object (e.g., VCR) 51, andsupplies the inputted visual ID 52 to the image processing unit 61 viathe PCMCIA interface 31.

In step S4, the image processing unit 61 identifies the object (e.g.,VCR) 51 based on the supplied visual ID 52, and supplies the identifiedobject data to the control unit 65.

In step S5, the user having the wearable unit 11 put on himself orherself inputs through the microphone 24 a voice (i.e., a voice memo) tobe attached to the editing studio and the VCR, and the inputted voicedata is supplied to the voice processing unit 63.

In step S6, the voice processing unit 63 executes a predeterminedprocess on the supplied voice data and supplies the processed voice datato the control unit 65, followed by proceeding to step S7.

In step S7, the control unit 65 transmits the supplied position data,object data and voice data to the server 12 via the communication unit32, followed by proceeding to step S8.

In step S8, the communication unit (not shown) of the server 12 receivesthe transmitted position data, object data and voice data via thewireless LAN 16.

In step S9, the server 12 registers the received voice data in thedatabase 12 a in correspondence to the position data and the objectdata, thereby completing the process.

The process of reading and displaying the information, which has beenattached to situations in the real world, from the database 12 a in theAugment-able Reality system shown in FIG. 3 will be described below withreference to a flowchart of FIG. 10.

First, when the user having the wearable unit 11 put on himself orherself moves to a predetermined room (e.g., the editing studio), the IRsensor 22 receives the IR signal transmitted from the IR beacon 17 andsupplies the received signal to the position information detecting unit62 via the interface box 25 in step S21.

In step S22, the position information detecting unit 62 detects, basedon the supplied signal, the position at a room level (e.g., the factthat the room is the editing studio), and supplies the detected positiondata to the control unit 65.

In step S23, the CCD video camera 21 takes in the visual ID (e.g.,two-dimensional bar code) 52 pasted onto the object (e.g., VCR) 51, andsupplies the inputted visual ID 52 to the image processing unit 61 viathe PCMCIA interface 31.

In step S24, the image processing unit 61 identifies the object (e.g.,VCR) 51 based on the supplied visual ID 52, and supplies the identifiedobject data to the control unit 65.

In step S25, the control unit 65 transmits the supplied position dataand object data to the server 12 via the communication unit 32, followedby proceeding to step S26.

In step S26, the communication unit (not shown) of the server 12receives the transmitted position data and object data via the wirelessLAN 16.

In step S27, the server 12 reads, from the database 12a, the voice datacorresponding to the received position data and object data, followed byproceeding to step S28.

In step S28, the server 12 transmits the read voice data to thesub-notebook PC 26 from the transmission unit, followed by proceeding tostep S29.

In step S29, the transmission unit 32 of the sub-notebook PC 26 receivesthe transmitted voice data via the wireless LAN 16, and supplies thereceived voice data to the control unit 65, followed by proceeding tostep S30.

In step S30, the control unit 65 produces an icon corresponding to thesupplied voice data, and supplies the created icon to the combining unit66.

In step S31, the combining unit 66 combines the created icon with theimage data of the current sight supplied from the CCD video camera 21,and displays a combined image on the HUD 27, thereby completing theprocess.

The case of accessing the database 12 a from normal computerenvironments will be described below. FIG. 11 shows one example of ascreen image when information of the database 12 a is acquired from amap on the WWW described by, e.g., the so-called Java applet. In thescreen image of FIG. 11, the user can also attach information tosituations (such as a position and an object) in the real world acquiredvia the WWW and displayed on the screen.

Further, the user can send an E-mail addressed to the situation of thereal world by utilizing the computer 42. For example, when an E-mail istransmitted from the computer 42 to a “room 3” as shown in FIG. 12, theserver 12 receives the E-main via the E-mail interface 14. Then, whenanother user having the wearable unit 11 put on himself or herself comesto the “room 3,” the contents of a message (e.g., “Today's meeting iscancelled.”) is displayed on the HUD 27 and informed to the other user.

While an IR beacon is employed to detect a position in the above firstembodiment, detection of the position can also be made by utilizing theGPS (Global Positioning System) or PHS (Personal Handyphone System).

Additionally, a providing medium for providing a computer program forexecuting the above processes in the first embodiment to users includesnot only an information recording medium such as a magnetic disk orCD-ROM, but also a transmission medium in the form of a networkemploying the Internet or digital satellites.

It is to be noted that the term “system” used in the first embodimentimplies an overall facility made up of a number of devices, units andthe like.

Next, a second embodiment of the present invention will be describedwith reference to FIGS. 13 to 20.

The second embodiment of the present invention relates to an informationprocessing system in which an ID already attached to an object in thereal world is read by an ID reader so that, based on the read ID, theuser can register and retrieve a bulletin board in the BBS, for example,on the Internet or PC communication services, can browse information ofthe object corresponding to the ID on the retrieved bulletin board, andcan write and exchange information of the object to which the ID isattached.

FIG. 13 shows one concrete system according to the second embodiment ofthe present invention. In this embodiment, a wearable sub-notebook PC isused as one example of a personal computer. The system shown in FIG. 13represents the case of employing a client PC 304 as one example of aninformation processing apparatus (i.e., a user terminal), a book 307 asone example of an object in the real world, a bar code 306 as oneexample of an ID attached to the book 307, a bar code reader 305 forreading the bar code 306 as one example of an ID reader for reading theID, the Internet as one example of a network 303, and a server 301including a hard disk 302 as one example of another informationprocessing apparatus, respectively. The configuration of FIG. 13 isshown by way of example and the present invention is not limited to theillustrated one.

In FIG. 13, the server 301 is similar to an ordinary WWW (World WideWeb) server, and comprises a main unit computer (server 301), the harddisk 302 for storing information of bulletin boards associated withobjects corresponding to IDs, a network connecting device (not shown)for connecting the server 301 to the network 303, etc.

Further, the client PC 304 comprises a sub-notebook PC including amodem, a PCMCIA interface, etc., and is connectable to the Internet orthe like via the modem. The client PC 304 is also connectable to the barcode reader 305 (possibly a CCD camera instead), which serves as the IDreader, via the PCMCIA interface. The device for reading the ID is notlimited to an independent standalone device such as the ID reader inFIG. 13 (e.g., the bar code reader), but may be such a device asassembled in the client PC 304. For example, in the case that a CCDcamera is assembled in the client PC 304, the CCD camera can be used asthe ID reader. As another example, when numerals or the like that arevisually readable by the user are employed as an ID attached to anobject in the real world, a keyboard (e.g., a keyboard of the client PC304) for entering the ID is also included in the category of the IDreader.

FIG. 14 shows one example of an initial screen image displayed on adisplay unit (e.g., a liquid crystal display) of the client PC 304 whenan ID (e.g., the bar code 306) already attached to an article (e.g., thebook 307) in the real world is taken in by an ID reader (e.g., the barcode reader 305) and bulletin boards present in the Internet server 301,for example, are searched based on the inputted ID. Note that FIG. 14shows one example of the initial screen image and the present inventionis not limited to the illustrated one.

Referring to FIG. 14, an ID entry field 311 is indicated in the initialscreen image. When the user wants to browse the bulletin board whichcarries information regarding the book 307 in the real world, such ascritics and details, the user operates the bar code reader 305 to readthe bar code 306 attached to the book 307 (or the user enters numeralsof the bar code from the keyboard), whereupon the numerals correspondingto the bar code 306 are inputted in the ID entry field 311 of theinitial screen image.

When the ID (numerals) is inputted in the ID entry field 311, the clientPC 304 accesses the server 301 in such a manner as to encode the ID intoa URL (Uniform Resource Locator) of the WWW. If the ID is recognized as,e.g., “9780123456789,” a URL is given below, for example:

-   -   http://www.aaa.com/cgi-bin/bbs.cgi?id=9780123456789

If the bulletin board corresponding to the ID is already registered inthe server 301 upon such an access to the server 301, the server 301displays an image of the bulletin board corresponding to the ID as shownin FIG. 15, by way of example, on the display screen of the client PC304. Note that FIG. 15 shows one example of the display screen image andthe present invention is not limited to the illustrated one.

More specifically, the screen image of the bulletin board shown in FIG.15 includes, e.g., an ID number field 312 for indicating the ID number(“9780123456789”); a bulletin board name field 313 for indicating thename of the bulletin board (“Bulletin board of book ‘AABBCC’” in thisexample); a link address field 314 for link to another search service(e.g., a link for on-line shopping); a mailing field 315 in which afield, buttons, etc. for new mailing to the bulletin board areindicated; and a registered information field 316 for indicatinginformation already registered on the bulletin board. Since the bulletinboard in the format shown in FIG. 15 enables users to browse andexchange information of the object (book) corresponding to the ID, eachuser can refer to opinions of others, retrieve a desired bulletin board,or write new information on the bulletin board as with the ordinary BBS,for example.

On the other hand, in some cases, when the server 301 is accessed with aURL corresponding to the ID, the ID is not yet registered in the server301 (the bulletin board corresponding to the ID is not present).

In such a case, the server 301 displays, on the display screen of theclient PC 304, a new registration screen image for prompting the user tomake new registration of a bulletin board corresponding to the ID asshown in FIG. 16, by way of example. Note that FIG. 16 shows one exampleof the display screen image and the present invention is not limited tothe illustrated one.

More specifically, the screen image for new registration shown in FIG.16 includes a name entry field 317 for prompting the user to enter basicinformation, e.g., the name of the article, corresponding to the ID(“9881234” in the example of FIG. 16). When the user inputs, in the nameentry field 317, the name of the article (“Health Drink XYZ” in theexample of FIG. 16) corresponding to the ID, a new bulletin boarddesignated by the ID (“9881234”) and “Health Drink XYZ,” i.e., the nameof the article corresponding to the ID, is registered in the server 301.

With the second embodiment of the present invention, as described above,based on the ID already attached to an article (object) in the realworld, the user can register and retrieve, e.g., a bulletin board in theInternet BBS, can browse information of the article corresponding to theID on the retrieved bulletin board, and can write and exchangeinformation of the article corresponding to the ID.

Of course, the second embodiment is not limited to the book or healthdrink described above, but is also applicable to other suitableproducts. By scanning a bar code attached to a music CD, for example, itis possible to access or jump to a bulletin board for the CD, and henceto easily acquire information regarding the music of the CD. Since thebulletin board for the CD can be accessed by other users who areinterested in the same CD, circulation of information and formation ofuser communities is promoted. Likewise, by scanning a bar code attachedto an electric product, for example, it is possible to obtain the latestinformation relating to the electric product from the user community.

In the system of the second embodiment, the information written on thebulletin board is not limited to information provided from the client PCside as described above, but may be, e.g., information of variousproducts marketed from the product selling side. When the productselling side offers the product information via the BBS, a place forexchanging information can be provided through a universal interfacefor, e.g., all types of products marketed from the product selling side.

In the second embodiment, the ID is assumed to be a typical bar code asdescribed above. This is because model numbers are printed or attachedin the form of bar codes onto most of products and they can be used asIDs as they are. There are other available ID systems including, e.g.,the JAN code (the EAN code in Europe), the ISBN code, and the CDDB ID.The JAN code is one type of bar code systems, and is printed in the formof a bar code on a product package for uniquely identifying a packagedproduct. The ISBN code is a code system for uniquely identifying a book,and the code number is printed on, e.g., a back cover of each book. Inmany cases, a bar code corresponding to the ISBN code is also printedalong with the ISBN code. The CDDB ID is an ID system used on theInternet for identifying CDs, and is already available at theURL/http:/www.cddb.com. More specifically, the CDDB ID is created byprocessing information of a music CD, such as the number of tracks andthe playing time, for conversion into numerals of 32 bits. The CDDB IDnumber can be used as an ID of each CD. Further, the name capable ofuniquely identifying an object can also be used as an ID in addition toa bar code. For example, the address of E-mail for identifying persons,the URL of a home page on the WWW, the phone number, the address, andother proper names such as the company name, the trade name, the name ofa store and the brand name, can be used as an ID.

Additionally, it is conceivable that a two-dimensional ID such as theso-called cybercode and a non-contact ID such as a wireless tag are usedas IDs. Depending upon the type of ID, the screen image can be linkedwith another information service, e.g., on-line shopping. The exampleshown in FIG. 15 represents a BBS screen image provided when scanningthe universal number (e.g., the ISBN code) of a book with a bar codereader, and includes not only information of the bulletin boardregarding the book, but also the link address, indicated in the linkaddress field 314, to an on-line book selling page.

FIG. 17 shows a process flow when on the client PC side in the systemaccording to the second embodiment, an ID attached to an article in thereal world is read by an ID reader, and retrieval and browsing of andwriting on a bulletin board corresponding to the ID are performed, or abulletin board is newly registered. Note that FIG. 17 shows one exampleof the process flow and the present invention is not limited to theillustrated one.

Referring to FIG. 17, in step S41, the client PC 304 first startsapplication software for running a BBS search with an ID according tothe second embodiment. In the second embodiment, since the client PC 304including a personal computer is employed as one example of the userterminal, the application software is started to run a BBS search instep S41. However, if the user terminal is a dedicated device forexclusively executing a BBS search and display with an ID, the dedicateddevice is powered on in step S41.

When the application software is started in step S41, the client PC 304waits in step S42 for an input of an article ID from the ID reader 305.If the article ID 306 is inputted from the ID reader 305, the client PC304 establishes connection to the server 301 in step S43, and transmitsthe inputted ID 306 to the server 301 in step S44 after encoding the IDinto a corresponding URL.

Then, in step S45, the client PC 304 receives a reply from the server301. The reply from the server 301 is provided as either information ofa bulletin board corresponding to the ID as shown in FIG. 15, or arequest for registration of a new bulletin board (registration of a newID) as shown in FIG. 16. Accordingly, the processing in the client PC304 advances to steps subsequent to S47 if the reply from the server 301is information of a bulletin board, and to steps subsequent to S52 ifthe reply from the server 301 is a request for registration of a newbulletin board.

If it is determined in step S46 that the reply from the server 301 isinformation of a bulletin board, the client PC 304 displays an imagescreen of the bulletin board corresponding to the ID as shown in FIG.15, by way of example, on the display screen in step S47.

On the other hand, if it is determined in step S46 that the reply fromthe server 301 is a request for registration of a new bulletin board,the client PC 304 goes to step S52 in which it displays, on the displayscreen, a new registration screen image for prompting the user to makenew registration of a bulletin board corresponding to the ID as shown inFIG. 16, and also indicates, in the screen image for new registration,the name entry field 317 for prompting the user to enter the name of anarticle corresponding to the ID.

Thereafter, in step S53, the client PC 304 transmits the name of thearticle entered in the name entry field 317 to the server 301. After thecompletion of step S53, the process flow returns to step S45. As aresult, the newly registered bulletin board is transmitted to the clientPC 304 from the server 301.

Then, the client PC 304 determines in step S48 whether there is anyinformation to be written on the bulletin board. If there is anyinformation to be written on the bulletin board, the client PC 304transmits the written information to the server 301 in step S49. Afterthe completion of step S49, the process flow goes to step S50.

If it is determined in step S48 that there is no information to bewritten on the bulletin board, the processing in the client PC 304advances to step S50.

In step S50, the client PC 304 determines whether there is an input of anew ID from the ID reader. If it is determined in step S50 that a new IDis inputted from another article, for example, the processing of theclient PC 304 returns to step S44, following which the new ID istransmitted to the server 301. On the other hand, if it is determined instep S50 that a new ID is not inputted, the process flow goes to stepS51.

If an input of a new ID is not determined in step S50 and cutoff of thecommunication line is instructed from the user, the client PC 304 cutsoff the communication line between itself and the server 301 in stepS51. After cutoff of the communication line in step S51, if it isdetermined in step S54 that the application has been brought into end orthe power has been turned off, the processing shown in the flowchart ofFIG. 17 is ended. If it is determined in step S54 that neither theapplication has been brought into end nor the power has been turned off,the process flow returns to step S42 in which the client PC 304 waitsfor an ID input. In the flowchart of FIG. 17, the communication linebetween the server 301 and the client PC 304 is connected at the time ofstarting transmission of the ID to the server 301, and the communicationline is cut off upon an instruction from the user to cut off it.However, the client PC 304 may be kept always connected to the server301.

FIG. 18 shows a process flow when, in response to an ID transmitted fromthe client PC 304, the bulletin board corresponding to the ID or arequest for new registration of a bulletin board is transmitted to theclient PC 304 from the side of the server 301 in the second embodiment.Note that FIG. 18 shows one example of the process flow and the presentinvention is not limited to the illustrated one.

Referring to FIG. 18, as shown in step S61, the server 301 is alwaysheld in a state waiting for reception of an ID from the client PC 304.If the server 301 receives an ID from the client PC 304 in step S61, theprocess flow goes to step S62.

After receiving an ID from the client PC 304 in step S61, the server 301determines in step S62 whether the received ID is registered, i.e.,whether the bulletin board corresponding to the received ID isregistered. If it is determined in step S62 that the bulletin boardcorresponding to the received ID is not registered, the processing inthe server 301 advances to steps subsequent to S66, whereas if it isdetermined in step S62 that the bulletin board corresponding to thereceived ID is registered, the processing in the server 301 advances tosteps subsequent to S63.

If it is determined in step S62 that the bulletin board corresponding tothe received ID is registered, the server 301 transmits information ofthe bulletin board corresponding to the ID to the client PC 304 in stepS63. After the completion of step S63, the process flow goes to stepS64.

On the other hand, it is determined in step S62 that the bulletin boardcorresponding to the received ID is not yet registered, the server 301transmits, to the client PC 304, information of the new registrationscreen image for prompting the user to make new registration of thebulletin board corresponding to the ID as shown in FIG. 16.

After receiving the name of an article corresponding to the ID from theclient PC 304 in step S67, the server 301 creates and registers, as thebulletin board corresponding to the received ID, a bulletin board forthe name of the article in step S68, and then transmits information ofthe newly registered bulletin board to the client PC 304 in step S69.After the completion of step S69, the process flow goes to step S64.

Then, the server 301 determines in step S64 whether new information iswritten on the bulletin board from the client PC 304, i.e., whether newwritten information is transmitted from the client PC 304. If it isdetermined in step S64 that no information is written on the bulletinboard, the processing in the server 301 returns to step S61 in which theserver 301 is held in the state waiting for reception of an ID from theclient PC 304.

On the other hand, if it is determined in step S64 that information iswritten on the bulletin board, the server 301 adds the writteninformation to the bulletin board or updates the information on thebulletin board depending upon the written information in step S65. Afterthe completion of step S64, the processing in the server 301 returns tostep S61 in which the server 301 is held in the state waiting forreception of an ID from the client PC 304.

The above second embodiment has been described in connection with anexample in which one bulletin board corresponds to one ID. However, whena number of ID systems including, e.g., the JAN code (the EAN code inEurope), the ISBN code, the CDDB ID, and wireless tags, as describedabove, are usable together in the information processing system, it mayhappen that the same ID is used among a number of different ID systems.In such a case, one ID is used double or more and represents a number ofdifferent articles. If a number of different articles have the same ID,this results in that when a bulletin board corresponds to that ID isretrieved, a number of bulletin boards for the number of differentarticles are found out by a search.

In view of the above, the system of this embodiment is designed suchthat when a number of bulletin boards are found corresponding to thesame ID in the server 301, the names, abstracts, etc. of the number ofbulletin boards are provided to the client PC 304 so that the user mayselect desired one from among the number of bulletin boards.

FIG. 19 shows a process flow on the side of the server when a number ofbulletin boards corresponding to the same ID are found out by a searchas described above. Concretely, the process flow of FIG. 19 can beeffected in, e.g., step S63 in the flowchart of FIG. 18. Note that FIG.19 shows one example of the process flow and the present invention isnot limited to the illustrated one.

Referring to FIG. 19, in step S71, the server 301 first determineswhether when an ID is transmitted from the client PC 304, a number ofbulletin boards are registered corresponding to the ID.

If it is determined in step S71 that a number of bulletin boards are notregistered corresponding to the ID, information of the bulletin boardcorresponding to the ID is transmitted to the client PC 304 in step S74.

On the other hand, if it is determined in step S71 that a number ofbulletin boards are registered corresponding to the ID, the server 301transmits, e.g., a list of the names, abstracts, etc. of the number ofbulletin boards to the client PC 304 in step S72.

After transmitting the list of the names, abstracts, etc. of the numberof bulletin boards to the client PC 304, upon receiving a selectionsignal for the name, etc. (i.e., a selection signal for the bulletinboard) which has been selected by the client PC 304 from among the listof the names, abstracts, etc. of the number of bulletin boards in stepS73, the server 301 transmits information of the bulletin boardcorresponding to the selection signal to the client PC 304 in step S74.

FIG. 20 shows a process flow on the side of the client PC 304 when thelist of the names, abstracts, etc. of the number of bulletin boards aretransmitted from the side of the server 301. Concretely, the processflow of FIG. 20 can be effected in, e.g., step S47 in the flowchart ofFIG. 17. Note that FIG. 20 shows one example of the process flow and thepresent invention is not limited to the illustrated one.

Referring to FIG. 20, in step S81, the client PC 304 first determineswhether the signal transmitted from the server 301 provides the list ofthe names, etc. of the number of bulletin boards.

If it is determined in step S81 that the signal transmitted from theserver 301 does not provide the list of the names, etc. of the number ofbulletin boards, i.e., that information of one bulletin boardcorresponding to the ID is transmitted from the server 301, the clientPC 304 displays the only one bulletin board transmitted from the server301 on the display screen in step S84.

On the other hand, if it is determined in step S81 that the signaltransmitted from the server 301 provides the list of the names, etc. ofthe number of bulletin boards, the client PC 304 displays the list ofthe names, etc. of the number of bulletin boards on the display screenin step S82, thereby prompting the user to select the desired name(i.e., the desired bulletin board) from among the list of the names,etc.

Then, in step S83, the client PC 304 transmits to the server 301 thename of the bulletin board that has been selected by the user from amongthe list of the names, etc. Thereafter, when information of the bulletinboard corresponding to the selected name, etc. is transmitted from theserver 301, the client PC 304 displays the bulletin board transmittedfrom the server 301 on the display screen in step S84.

In the case of employing a number of different ID systems together, thedouble or more use of the same ID can be avoided by managing the BBS insuch a manner, for example, that the name (e.g., ISBN) of the ID systemis added to each ID. More specifically, when the client PC 304 takes inan ID, it transmits the ID to the server 301 along with informationindicating to which system the ID belongs (i.e., the name of the IDsystem). The server 301 searches bulletin boards based on both the IDand the information (the name of the ID system) added to the ID. Withsuch a management, even if the same ID is employed in different IDsystems, one bulletin board can be retrieved corresponding to one ID.

Additionally, a providing medium for providing a computer program forexecuting the above processes in the second embodiment to users includesnot only an information recording medium such as a magnetic disk orCD-ROM, but also a transmission medium in the form of a networkemploying the Internet or digital satellites.

It is to be noted that the term “system” used in the second embodimentimplies an overall facility made up of a number of devices, units andthe like.

Next, a third embodiment of the present invention will be described withreference to FIGS. 21 to 28.

While an article in the real world (in practice, an ID attached to thearticle) is handled in correspondence to a bulletin board in the secondembodiment, a position in the real world is handled in correspondence toa bulletin board in the third embodiment.

Specifically, the third embodiment relates to an information processingsystem in which a current position in the real world is taken in by aposition recognizing device, and a bulletin board in the BBS on theInternet or PC communication services, for example, can be registeredbased on the inputted current position, which enables the user toretrieve and browse bulletin boards corresponding to the inputtedcurrent position and nearby positions, and which enables the user towrite new information or to exchange information on and via the bulletinboards corresponding to the inputted current position and nearbypositions.

FIG. 21 shows one concrete system according to the third embodiment ofthe present invention. In this embodiment, a sub-notebook PC is used asone example of a user terminal and is referred to as a client PC 404hereinafter. The system shown in FIG. 21 represents the case ofemploying a portable client PC 404 as one example of an informationprocessing apparatus (i.e., a user terminal), a position recognizingdevice 405 such as utilizing the GPS or PHS as one example of a devicefor detecting current position information in the real world, a wirelessnetwork and the Internet as one example of a network 403, and a server401 including a hard disk 402 as one example of another informationprocessing apparatus, respectively. Also, the client PC 404 includes aninformation input device 406 for inputting information (e.g., a text, aphotograph and/or a voice memo) related to the current position. Theuser terminal may include the wearable computer 8 as with the firstembodiment. The configuration of FIG. 21 is shown by way of example andthe present invention is not limited to the illustrated one.

In FIG. 21, the server 401 is similar to the WWW server in FIG. 13, andincludes a main unit computer (server 401), the hard disk 402 forstoring information of, e.g., bulletin boards corresponding torespective positions in the real world, a network connecting device (notshown) for connecting the server 401 to the network 403, etc.

The user terminal (the client PC 404) includes a sub-notebook PCincluding a modem, a PCMCIA interface, etc., and is connected via thePCMCIA interface to a wireless network device 407 (including a cellularphone and PHS) and the position recognizing device 405 (such asutilizing the GPS or PHS). Further, the client PC 404 includes a CCDcamera and a microphone as one example of the information input device406 for inputting information related to the current position, and aninternal clock for recognizing the current time. The CCD camera and themicrophone serving as the information input device 406 may be integralwith the client PC 404 or separate from the same. The client PC 404 cantransmit current position information detected by the positionrecognizing device 405, current time information from the internalclock, and information related to the current position, such as aphotograph and a voice memo inputted from the information input device406 or text data inputted from a keyboard or the like, to the server 401via the wireless network device 407 and the network 403.

In the system of the third embodiment, for example, when the bulletinboard corresponding to the current position is not registered in theserver 401 and is newly registered, the client PC 404 receivesinformation (e.g., a photograph, a text and/or a voice mail) related tothe current position from the information input device 406 or thekeyboard, and transmits the received information to the server 401 afteraffixing, as search tags, the current position information detected bythe position recognizing device 405 and the current time informationfrom the internal clock to the information related to the currentposition. As a result, the bulletin board corresponding to the currentposition is registered in the server 401.

Also, in the system of the third embodiment, when the bulletin boardcorresponding to the current position is registered in the server 401and new information is additionally written on the bulletin boardcorresponding to the current position from the client PC 404, the clientPC 404 acquires the new information from the information input device406 or the keyboard, and transmits the acquired information to theserver 401 after affixing, as search tags, the current positioninformation detected by the position recognizing device 405 and thecurrent time information from the internal clock to the informationrelated to the current position. As a result, the server 401 adds orupdates the new information related to the current position to or on thebulletin board corresponding to the current position.

Moreover, in the system of the third embodiment, when the bulletin boardregistered in the server 401 is retrieved or browsed from the client PC404, the client PC 404 transmits to the server 401 the current positioninformation detected by the position recognizing device 405 andadditional search conditions including a keyword inputted from, e.g.,the keyboard, and the current time information from the internal clock.Based on the transmitted current position information and additionalsearch conditions, the server 401 retrieves the target bulletin boardfrom the hard disk 402 and transmits information of the retrievedbulleting board to the client PC 404.

FIG. 22 shows one example of a bulletin board transmitted from theserver 401 (i.e., one example of a display screen image on the client PC404) when the current position is in front of some restaurant having thespecific name of “Restaurant AA,” for example, and informationinstructing the name of “Restaurant AA” and the bulletin board beingcreated within “7 days” from the current time is inputted as theadditional search conditions from the client PC 404. The display screenimage of FIG. 22 is shown by way of example and the present invention isnot limited to the illustrated one.

The display screen image of FIG. 22 includes a search key entry field411 in which an additional search condition related to the position isinputted, and a time key entry field 412 in which a desired period oftime from the current time (date and time) is inputted as anotheradditional search condition. When “Restaurant AA” is inputted as akeyword in the search key entry field 411 and “7 days” is inputted as akeyword in the time key entry field 412, a photograph 413 of theRestaurant AA and texts 414, 415 (e.g., “Hamburger steak is delicious”and “Spaghetti is unpalatable”) related to comments about the RestaurantAA are displayed as one example of the bulletin board transmitted fromthe server 401 corresponding to the inputted keywords.

FIG. 22 shows an example of the display screen image when a search isrun by inputting a specific keyword, such as “Restaurant AA,” at thecurrent position and the bulletin board for “Restaurant AA” was alreadyregistered in the past in relation to the current position. It ishowever very rare that the current position completely coincides withthe position at which some bulletin board was registered in the past.For this reason, when a search is run in the system according to thethird embodiment of the present invention, bulletin boards matching theadditional search conditions are also retrieved from among the bulletinboards which have been already registered at respective positions nearthe current position. In such a case, it is desired in the example ofFIG. 22 that the distance and direction from the current position to“Restaurant AA” are indicated on the display screen imagesimultaneously. Further, when a search is run by inputting, as aspecific keyword, a general name, e.g., “restaurant,” a number ofbulletin boards which are associated with “restaurant” and have beenalready registered at respective positions near the current position areretrieved.

FIG. 23 shows one example of a display screen image when bulletin boardsmatching with the additional search conditions are retrieved by theserver 401 from among the bulletin boards corresponding to respectivepositions near the current position as described above, and the bulletinboards corresponding to the nearby positions are displayed on thedisplay screen of the client PC 404. The display screen image of FIG. 23is shown by way of example and the present invention is not limited tothe illustrated one.

Referring to FIG. 23, the display screen image on the client PC 404includes a distance scale 425 having the center O corresponding to thecurrent position and indicating the distance from the center O (thecurrent position), as well as the search key entry field 411 and thetime key entry field 412. In the example of FIG. 23, when a generalname, e.g., “restaurant,” is inputted as a keyword in the search keyentry field 411 and “7 days” is inputted as a keyword in the time keyentry field 412, bulletin boards 421, 422, 423 and 424 matching thoseadditional search conditions are found from among a number of bulletinboards which have already been registered at respective positions nearthe center O (the current position). The bulletin boards 421, 422, 423and 424 are then displayed on the display screen of the client PC 404 asshown in FIG. 23. While the distance within the radius of 200 m from thecenter O is set as a search range in the example of FIG. 23, the searchrange may be set to a larger or smaller distance.

With the third embodiment of the present invention, as described above,when the bulletin board corresponding to the current position isretrieved, a number of bulletin board corresponding to the currentposition and nearby positions are also obtained. Therefore, the user canbrowse, for example, opinions of other persons who visited therespective positions in the past.

When running a BBS search as described above, filtering of bulletinboards can also be performed based on other attendant information (e.g.,time, creator, and keyword explicitly attached to the bulletin board).One example of filtering based on time is as follows. By registeringphotographs and messages about climbing of Mt. Fuji, for example, as abulletin board in the server, the user can retrieve the bulletin board,which includes the desired photograph, with such an additional searchcondition as “prospect seen from the same place in winter” when the userclimbs Mt. Fuji in summer. This makes the user's experience of mountainclimbing more plentiful. In streets as another example, the user canattach such information as “impression of this restaurant” to thebulletin board corresponding to each restaurant or shop, and can browsethe bulletin boards corresponding to the respective restaurants orshops. Therefore, users can form a useful set of guide book informationwhile also participating themselves.

FIGS. 24 and 25 show process flows when on the side of the client PC 404in the system of the third embodiment, a bulletin board containinginformation related to the current position is newly registered, and,for example, retrieval and browsing of and writing on a bulletin boardalready registered corresponding to the current position are performed.Specifically, FIG. 24 primarily shows the process flow for newregistration of a bulletin board, and FIG. 25 primarily shows theprocess flow for, e.g., retrieval and browsing of and writing on abulletin board. The process flows of FIGS. 24 and 25 are shown by way ofexample and the present invention is not limited to the illustratedones.

Referring to FIG. 24, in step S91, the client PC 404 first startsapplication software for executing registration, retrieval, etc. of abulletin board corresponding to the current position in the thirdembodiment described above. In the third embodiment, since a personalcomputer (the client PC 404 in FIG. 21) is employed as one example ofthe user terminal, the application software is started to executeregistration, retrieval, etc. of a bulletin board corresponding to thecurrent position in step S91. However, if the user terminal is adedicated device for exclusively executing registration, retrieval, etc.of a bulletin board corresponding to the current position, the dedicateddevice is powered on in step S41.

When the application software is started in step S91, the client PC 304determines in step S92 whether an instruction from the user indicatesnew registration or retrieval of a bulletin board. If it is determinedin step S92 that retrieval of a bulletin board is instructed, theprocess flow goes to step S102 for shift to a bulletin board search (BBSsearch) routine shown in FIG. 25. On the other hand, if it is determinedin step S92 that new registration of a bulletin board is instructed, theprocess flow goes to steps subsequent to S93.

In step S93, the client PC 404 acquires or creates data, such as aphotograph, a voice, a hand-written memo and/or a text, from theinformation input device or using the keyboard. The data may be createdby or acquired from another application.

Further, the client PC 404 acquires current position information fromthe position recognizing device in step S94 and current time (date andtime) information from the internal clock in step S95. Thereafter, instep S96, the client PC 404 adds the current position information andthe current time information acquired respectively in steps S94 and S95,as search tags, to the data acquired or created in step S93.

Then, the client PC 404 establishes a connection to the server 401 instep S97, and transmits the data along with the search tags affixed toit to the server 401 in step S98. As a result, the bulletin boardcorresponding to the current position is newly registered in the server401.

After that, the client PC 404 determines in step S99 whether a bulletinboard search is instructed from the user. If it is determined in stepS99 that a bulletin board search is instructed, the process flow goes tostep S102 for shift to the bulletin board search routine shown in FIG.25. On the other hand, if it is determined in step S99 that a bulletinboard search is not instructed, the processing in the client PC 404 goesto step S100.

If cutoff of the communication line is instructed from the user, theclient PC 404 cuts off the communication line between itself and theserver 401 in step S100. After cutoff of the communication line in stepS100, it is determined in step S101 whether the application has beenbrought into end or the power has been turned off. If it is determinedthat neither the application has been brought into end nor the power hasbeen turned off, the process flow returns to step S92. If it isdetermined that the application has been brought into end or the powerhas been turned off, the processing shown in the flowchart of FIG. 24 isended. In the flowchart of FIG. 24, the communication line between theserver 401 and the client PC 404 is connected at the time of startingtransmission of the data and the search tags to the server 401, and thecommunication line is cut off upon an instruction from the user to cutit off. However, the client PC 404 may be kept always connected to theserver 401.

If the process flow goes to step S102 upon such a determination in stepS92 of FIG. 24 that new registration of a bulletin board is instructed,the client PC 404 acquires current position information from theposition recognizing device in step S111 in FIG. 25, and then acquires,as additional search conditions, a keyword from, e.g., the keyboard andcurrent time (date and time) information from the internal clock in stepS112.

Subsequently, the client PC 404 establishes a connection to the server401 in step S113, and transmits the current position and the additionalsearch conditions in step S114, followed by receiving a reply from theserver 401 in step S115.

It is here that the reply from the server 401 is provided as either abulletin board retrieved based on the current position and theadditional search conditions both transmitted from the client PC 404, ora message, for example, issued in the case that a search has ended infailure.

Accordingly, the processing in the client PC 404 advances to stepssubsequent to S117 if it is determined in step S116 that the reply fromthe server 401 is information of a bulletin board, and to stepssubsequent to S122 if it is determined in step S116 that the reply fromthe server 401 is a message, for example, indicating a search failure.

If it is determined in step S116 that the reply from the server 401indicates a search failure, the client PC 404 determines in step S122whether a new additional search condition has been acquired. If it isdetermined that a new additional search condition has not been acquired,the processing of the client PC 404 advances to step S121. If it isdetermined that a new additional search condition has been acquired, theprocessing of the client PC 404 advances to step S123 in which the newadditional search condition is transmitted to the server 401, followedby returning to step S115.

On the other hand, if it is determined in step S116 that the reply fromthe server 401 is information of a bulletin board, the client PC 404displays one or more bulletin boards (corresponding to the currentposition or nearby positions as described above) transmitted from theserver 401 on the display screen in step S117.

Then, the client PC 404 determines in step S118 whether there is anyinformation to be written on the bulletin board. If there is anyinformation to be written, the client PC 404 transmits the writteninformation to the server 401 in step S119. After the completion of stepS119, the processing in the client PC 404 advances to step S120.

If it is determined in step S118 that there is no information to bewritten on the bulletin board, the processing in the client PC 404advances to step S120.

In step S120, the client PC 404 determines whether there is an input ofa new current position and additional search condition. If it isdetermined in step S120 that a new current position and additionalsearch condition are inputted, the processing in the client PC 404returns to step S114 in which the new current position and additionalsearch condition are transmitted to the server 401. On the other hand,if it is determined in step S120 that a new current position andadditional search condition are not inputted, the process flow goes tostep S121.

If a new current position and additional search condition are notinputted in step S120 and cutoff of the communication line is instructedfrom the user, the client PC 404 cuts off the communication line betweenitself and the server 401 in step S121. After cutoff of thecommunication line in step S121, the process flow goes to step S101 inFIG. 24. If it is determined in step S101 that the application has beenbrought into end or the power has been turned off, the processing isended. If it is determined in step S101 that neither the application hasbeen brought into end nor the power has been turned off, the processflow returns to step S92 in FIG. 24 in which the client PC 404 waits foran instruction of a retrieval or new registration of a bulletin board.In the flowchart of FIG. 25, the communication line between the server401 and the client PC 404 is connected at the time of startingtransmission of the current position and the additional searchconditions to the server 401, and the communication line is cut off uponan instruction from the user to cut off it. However, the client PC 404may be kept always connected to the server 401.

FIGS. 26 and 27 show process flows when on the side of the server 401 inthe system of the third embodiment, a request for retrieval of abulletin board or data for new registration of a bulletin board istransmitted from the client PC 404, and retrieval or new registration ofa bulletin board is run correspondingly. Specifically, FIG. 26 primarilyshows the process flow for new registration of a bulletin board, andFIG. 27 primarily shows the process flow for retrieval of a bulletinboard, etc. The process flows of FIGS. 26 and 27 are shown by way ofexample and the present invention is not limited to the illustratedones.

Referring to FIG. 26, as shown in step S131, the server 401 is alwaysheld in a state of waiting for a request for retrieval of a bulletinboard or data for new registration of a bulletin board from the clientPC 404. If it is determined in step S131 that a request for retrieval ofa bulletin board is transmitted from the client PC 404, the process flowgoes to step S134 for shift to a bulletin board search routine shown inFIG. 27. On the other hand, if it is determined in step S131 that datafor new registration of a bulletin board is transmitted from the clientPC 404, the process flow goes to steps subsequent to S132.

In step S132, the server 401 creates a bulletin board corresponding tothe data, such as a photograph, a voice, a hand-written memo and/or atext, transmitted from the client PC 404, as well as the search tags,i.e., the current position and the current time (date and time), affixedto the data.

Then, in step S133, the server 401 stores the thus-created bulletinboard in the hard disk 402. The registration process of a new bulletinboard is thereby completed. After step S133, the server 401 returns tothe state of waiting for reception of a signal in step S131.

If the process flow goes to step S134 upon such a determination in stepS131 of FIG. 26 that new registration of a bulletin board is instructed,the server 401 receives, in step S141 of FIG. 27, the current positionand the additional search conditions both transmitted from the client PC404.

After receiving the current position and the additional searchconditions from the client PC 404, the server 401 searches in step S142the bulletin boards, which are stored in the hard disk 402, for onematching the current position and the additional search conditions. If abulletin board in match with the current position and the additionalsearch conditions is found out by the search in step S142, theprocessing in the server 401 advances to steps subsequent to S143,whereas if not found out, the processing in the server 401 advances tosteps subsequent to S146.

If it is determined in step S142 that the search has finishedsuccessfully, the server 410 transmits the found-out bulletin board tothe client PC 404 in step S143.

Then, the server 401 determines in step S144 whether there is anyinformation written on the bulletin board from the client PC 404. If anyinformation is written, the server 401 adds the written information tothe bulletin board or updates the bulletin board with the writteninformation in step S145, followed by returning to step S131 in FIG. 26.

On the other hand, if it is determined in step S142 that the search hasended in failure, the server 410 transmits a search failure message, forexample, to the client PC 404. Thereafter, if a new additional searchcondition is received from the client PC 404 in step S147, the server401 runs a search again in step S142 using the new additional searchcondition. If a new additional search condition is not transmitted, theprocess flow returns to step S131 in FIG. 26.

When position recognition is performed by utilizing the PHS in the thirdembodiment of the present invention, one conceivable method is tocalculate the latitude and the longitude based on the principles oftriangular surveying by using the intensities of electric fieldsgenerated from base stations whose positions are known. As anothermethod, it is also conceivable to employ, as a “position” ofconvenience, an ID of the base station generating an electric field withthe maximum intensity, or to employ, as a “position” of convenience, apair of base station ID and electric field intensity (possibly pluralpairs depending upon cases). The reason why the term “of convenience” isused here is that there is no direct relation between the base stationID and the latitude and longitude. In the third embodiment of thepresent invention, since it is enough to be able to determine whether“positions” corresponding to registered new bulletin boards are close toeach other, absolute position information is not necessarily required.

In the method of employing, as a “position” of convenience, a pair ofbase station ID and electric field intensity, a “distance” ofconvenience can be calculated with similarity calculation made betweenone pair and another pair of base station ID and electric fieldintensity which is observed later. A computing process shown in FIG. 28,by way of example, can be used as a similarity determining algorithm forexecuting the similarity calculation. The computing process shown inFIG. 28 corresponds to calculation of correlation coefficients among aset of base station IDs/electric field intensities. Additionally, inFIG. 28, Di represents the ID of the i-th base station, Fi representsthe electric field intensity of the i-th base station, and d representsthe “distance” of convenience between [IDi, Fi] (i=1, n) and [ID′j, F′j](j=1, . . . , m).

Referring to FIG. 28, the client PC 404 first initializes a variable“distance d” to zero (0) in step S151, and starts calculating the“distance d” of convenience between [IDi, Fi] (i=1, . . . , n) and[ID′j, F′j] (j=1, 2, . . . m) in step S152.

After starting the calculation, the client PC 404 determines in stepS153 whether a calculation loop for i, j at i=1, 2, . . . n and j=1, 2,. . . m is completed. If it is determined in step S153 that thecalculation loop is not yet completed, the client PC 404 determines instep S154 whether IDi=ID′j is satisfied. If it is determined in stepS154 that IDi=ID′j is not satisfied, the process flow returns to stepS153, whereas if it is determined that IDi=ID′j is satisfied, theprocess flow returns to step S153 after setting d=d+Fi*F′j in step S155.

Then, it is determined in step S153 that the calculation loop iscompleted, the client PC 404 sets d=d/(m*n) and obtains the distance d.

While the above-described third embodiment utilizes the GPS or PHS fordetecting a position, it is also possible to utilize, e.g., trafficinformation beacons which have been recently installed for a trafficinformation providing system, or an IR beacon used in the firstembodiment for the same purpose.

Additionally, a providing medium for providing a computer program forexecuting the above processes in the third embodiment to users includesnot only an information recording medium such as a magnetic disk orCD-ROM, but also a transmission medium in the form of a networkemploying the Internet or digital satellites.

It is to be noted that the term “system” used in the third embodimentimplies an overall facility made up of a number of devices, units andthe like.

FIG. 29 shows one example of a concrete configuration of principal partsof client PCs (304, 404) applicable to the second and third embodiments.

Referring to FIG. 29, an ID taken in by the ID reader 305 such as a barcode reader is supplied to an ID analyzing unit 502 via a PCMCIAinterface 501. The ID analyzing unit 502 analyzes the supplied ID, andthen supplies the analyzed ID data to a control unit 503.

A position signal detected by the position recognizing device 405 suchas utilizing the GPS or PHS is supplied to a position informationcomputing unit 505 via a PCMCIA interface 504. The position informationcomputing unit 505 computes, from the supplied signal, the currentaltitude and longitude (in the case utilizing the GPS) orposition-of-convenience information (in the case utilizing the PHS)based on base station IDs, and then supplies the computed currentposition data to the control unit 503.

Data such as a photograph, a voice and/or a hand-written memo isinputted from the information input device 406, and the input data issupplied to an input information processing unit 506. The inputinformation processing unit 506 processes the input data depending uponthe types of data (such as a photograph, a voice or a hand-writtenmemo), and then supplies the processed data to the control unit 503.

Other data, e.g., a text or a keyword as a search condition, is enteredfrom a keyboard, etc. 508, and an internal clock 510 generates data ofcurrent time (date and time). The data from the keyboard, etc. 508 andthe data from the internal clock 510 are also supplied to the controlunit 503.

The control unit 503 creates a URL necessary for accessing the serverbased on the supplied ID data as described above in the secondembodiment, or information necessary for retrieval of a bulletin boardcorresponding to a position, registration of a new bulletin board, etc.based on the input data, the keyword, the current time data, etc. asdescribed above in the third embodiment. Further, the control unit 503creates a display signal for providing a screen image on an LCD (liquidcrystal display) 507 as a display unit, and transmits or receives datato or from the server, etc. via a communication unit 509 which serves asa device for connection to the modem or the wireless network.

With the second and third embodiments of the present invention, asdescribed above, since information can be easily exchanged usingbulletin boards corresponding to an object and position in the realworld, the user can obtain the latest information about a certain objectand position, and can easily inform other users of new information.Also, with the second and third embodiments of the present invention,the present system can also be utilized as an information system forpresenting product support information, advertisement information abouta particular place, and so on. Moreover, links (such as advertisementsof associated products or shops) related to an object or positioncorresponding to some bulletin board can be additionally displayed onthe bulletin board. Thus, the system of the present invention is alsovaluable as an advertisement medium for distributing information to acommunity of users who are interested in a certain object or position.

According to the information processing apparatus and method, theinformation processing system, and the providing medium of the presentinvention, as fully described above, since attached information isinputted as corresponding to specific information specifying a situationin the real world, the attached information can be virtually attached tothe situation in the real world. Also, the attached information isinformation that is related to the situation in the real world and canbe browsed and exchanged among users. According to the presentinvention, therefore, information (such as image information, textinformation and voice information) corresponding to situations (such asa position and an object) in the real world can be presented to users.In addition, users can not only attach the information to the situationsin the real world, but also quickly and easily find out desiredinformation.

Further, according to the information processing apparatus and methodand the providing medium of the present invention, since attachedinformation is stored in correspondence to a situation in the realworld, the information corresponding to the situation in the real worldcan be provided to users. According to the present invention, therefore,information (such as image information, text information and voiceinformation) corresponding to situations (such as a position and anobject) in the real world can be presented to users. In addition, userscan not only attach the information to the situations in the real world,but also quickly and easily find out desired information.

It should be understood that various changes and modifications to thepresently preferred embodiments described herein will be apparent tothose skilled in the art. Such changes and modifications can be madewithout departing from the spirit and scope of the present subjectmatter and without diminishing its intended advantages. It is thereforeintended that such changes and modifications be covered by the appendedclaims.

1. An information processing apparatus comprising: a display device; aspecific information input unit; an input device; a transmitter; areceiver; a processor; and a memory device storing instructions whichwhen executed by the processor cause the processor to operate with thedisplay device, the specific information input unit, the input device,the transmitter and the receiver to: (a) receive specific information;(b) specify a situation in a physical environment based on the receivedspecific information; (c) receive first attachment information relatingto the specified situation in the physical environment; (d) attach thefirst attachment information to the specified situation in the physicalenvironment; (e) transmit, to a second information processing apparatus,the first attachment information and the specific information; (f)receive second attachment information from the second informationprocessing apparatus, the second attachment information: relating to thespecified situation in the physical environment; and (ii) correspondingto said transmitted specific information; and (g) display said receivedsecond attachment information.
 2. The information processing apparatusof claim 1, wherein: (a) the specific information input unit includes animage input unit; and (b) the specific information includes an image ofthe physical environment.
 3. The information processing apparatus ofclaim 1, wherein: (a) the specific information input unit includes aposition detecting unit; and (b) the specific information includes aposition in the physical environment.
 4. The information processingapparatus of claim 1, wherein: (a) the specific information input unitincludes an object identifying unit; and (b) the specific informationincludes an object in the physical environment.
 5. The informationprocessing apparatus of claim 1, wherein when executed by the processor,the instructions cause the processor to operate with the input device toenable a user to input a search condition, said second attachmentinformation being selected based on said search condition.
 6. Theinformation processing apparatus of claim 1, wherein said secondattachment information is selected based on a position in the physicalenvironment.
 7. The information processing apparatus of claim 1, whereinthe first attachment information includes at least one of voiceinformation, image information, and text information.
 8. The informationprocessing apparatus of claim 1, wherein: (a) the situation in thephysical environment is associated with an object in the physicalenvironment; (b) the specific information includes an ID associated withthe object; and (c) the second attachment information includesinformation which: (i) is related to the object; and (ii) can be browsedand exchanged.
 9. The information processing apparatus of claim 1,wherein: (a) the situation in the physical environment is associatedwith a position in the physical environment; (b) the specificinformation includes position information of the physical environment;and (c) the received second attachment information includes informationwhich: (i) is related to the position; and (ii) can be browsed andexchanged.
 10. A method of operating an information processing apparatusincluding instructions, the method comprising: (a) causing a processorto execute the instructions to cause a specific information input unitto receive specific information; (b) causing the processor to executethe instructions to specify a situation in a physical environment; (c)causing the processor to execute the instructions to receive firstattachment relating to the specified situation in the physicalenvironment; (d) causing the processor to execute the instructions toattach the first attachment information to the specified situation inthe physical environment; (e) causing the processor to execute theinstructions to cause a transmitter to transmit, to a second informationprocessing apparatus, the first attachment information and the specificinformation; (f) causing the processor to execute the instructions tocause a receiver to receive second attachment information from thesecond information processing apparatus, the second attachmentinformation: (i) relating to the specified situation in the physicalenvironment; and (ii) corresponding to said transmitted specificinformation; and (g) causing the processor to execute the instructionsto cause a display device to display said received second attachmentinformation.
 11. The method of claim 10, wherein: (a) the specificinformation input unit includes an image input unit; and (b) thespecific information includes an image of the physical environment. 12.The method of claim 10, wherein: (a) the specific information input unitincludes a position detecting unit; and (b) the specific informationincludes a position in the physical environment.
 13. The method of claim10, wherein: (a) the specific information input unit includes an objectidentifying unit; and (b) the specific information includes an object inthe physical environment.
 14. The method of claim 10, which includescausing the processor to execute the instructions to enable a user toinput a search condition, said second attachment information beingselected based on said search condition.
 15. The method of claim 10,wherein said second attachment information is further selected based ona position in the physical environment.
 16. The method of claim 10,wherein the first attachment information includes at least one of voiceinformation, image information, and text information.
 17. The method ofclaim 10, wherein: (a) the situation in the physical environment isassociated with an object in the physical environment; (b) the specificinformation includes an ID associated with the object; and (c) thesecond attachment information includes information which: (i) is relatedto the object in the physical environment; and (ii) can be browsed andexchanged.
 18. The method of claim 10, wherein: (a) the situation in thephysical environment is associated with a position in the physicalenvironment; (b) the specific information includes position informationof the physical environment; and (c) the received second attachmentinformation includes information which: (i) is related to the position;and (ii) can be browsed and exchanged.
 19. A computer readable-mediumstoring instructions structured to cause an information processingapparatus to: (a) receive specific information; (b) specify a situationin a physical environment based on the received specific information;(c) receive first attachment information relating to the specifiedsituation in the physical environment; (d) attach the first attachmentinformation to the specified situation in the physical environment; (e)transmit, to a second information processing apparatus, the firstattachment information and the specific information; (f) receive secondattachment information from the second information processing apparatus,the second attachment information: (i) relating to the specifiedsituation in the physical environment; and (ii) corresponding to saidtransmitted specific information; and (g) display said received secondattachment information.
 20. An information processing apparatuscomprising: a receiver; a transmitter; a processor; and a memory devicestoring instructions which when executed by the processor, cause theprocessor to operate with the transmitter and the receiver to: (a)receive, from a second information processing apparatus: (i) specificinformation for specifying a situation in a physical environment; and(ii) first attachment information attached to the specified situation inthe physical environment, said first attachment information relating tothe specified situation in the physical environment; (b) store thereceived specific information and first attachment information; (c) readsecond attachment information that corresponds to the received specificinformation; and (d) transmit, to the second information processingapparatus, the second attachment information relating to the specifiedsituation in the physical environment.
 21. The information processingapparatus of claim 20, wherein the specific information includes atleast one of image information of the physical environment, positioninformation of the physical environment, and object information of thephysical environment.
 22. The information processing apparatus of claim20, wherein when executed by the processor, the instructions cause theprocessor to select the second attachment information based on areceived search condition.
 23. The information processing apparatus ofclaim 20, wherein when executed by the processor, the instructions causethe processor to select the second attachment information based on aposition in the physical environment.
 24. The information processingapparatus of claim 20, wherein: (a) the situation in the physicalenvironment is associated with an object in the physical environment;(b) the specific information includes an ID associated with the object;and (c) the transmitted second attachment information: (i) is related tothe object; and (ii) can be browsed and exchanged.
 25. The informationprocessing apparatus of claim 20, wherein: (a) the situation in thephysical environment is associated with a position in the physicalenvironment; (b) the specific information includes position informationof the physical environment; and (c) the transmitted second attachmentinformation: (i) is related to the position in the physical environment;and (ii) can be browsed and exchanged.
 26. A method of operating aninformation processing apparatus including instructions, the methodcomprising: (a) causing a processor to execute the instructions to causea receiver to receive, from a second information processing apparatus:(i) specific information for specifying a situation in a physicalenvironment; and (ii) first attachment information attached to thespecified situation in the physical environment, said first attachmentinformation relating to the specified situation in the physicalenvironment; (b) causing the processor to execute the instructions tostore the received specific information and attachment information; (c)causing the processor to execute the instructions to read secondattachment information that corresponds to the received specificinformation; and (d) causing the processor to execute the instructionsto cause a transmitter to transmit, to the second information processingapparatus, the second attachment information which relates to thespecified situation in the physical environment.
 27. The method of claim26, wherein the specific information includes at least one of imageinformation of the physical environment, position information of thephysical environment, and object information of the physicalenvironment.
 28. The method of claim 26, which includes causing theprocessor to execute the instructions to select the second attachmentinformation based on a position in the physical environment.
 29. Themethod of claim 26, which includes causing the processor to execute theinstructions to select the second attachment information based on areceived search condition.
 30. The method of claim 26, wherein: (a) thesituation in the physical environment is includes an object in thephysical environment; (b) the specific information includes an IDattached to the object in the physical environment; and (c) thetransmitted second attachment information includes information that: (i)is related to the object in the physical environment; and (ii) can bebrowsed and exchanged.
 31. The method of claim 26, wherein: (a) thesituation in the physical environment includes a position in thephysical environment; (b) the specific information includes positioninformation of the physical environment; and (c) the transmitted secondattachment information includes information that: (i) is related to theposition in the physical environment; and (ii) can be browsed andexchanged.
 32. A computer readable-medium storing instructionsstructured to cause an information processing apparatus to: (a) receive,from a second information processing apparatus: (i) specific informationfor specifying a situation in a physical environment; and (ii) firstattachment information attached to the specified situation in thephysical environment, said first attachment information relating to thespecified situation in the physical environment; (b) store the receivedspecific information and first attachment information; (c) read secondattachment information that corresponds to the received specificinformation; and (d) transmit, to the second information processingapparatus, the second attachment information relating to the specifiedsituation in the physical environment.
 33. An information processingsystem comprising: a first information processing apparatus; and asecond information processing apparatus, the first informationprocessing apparatus including: (a) a specific information input unitfor inputting specific information to specify a situation in the realworld; (b) an attachment information input unit for inputting attachmentinformation to be attached to the situation in the real world specifiedby the specific information; (d) a first transmitting unit fortransmitting the attachment information and the specific information tothe second information processing apparatus; (e) a searching unit forsearching the second information processing apparatus to acquire furtherattachment information based on further specific information inputted bythe attachment information input unit; (f) a first receiving unit forreceiving the further attachment information transmitted from the secondinformation processing apparatus; and (g) a display controlling unit forcontrolling to display the further attachment information received bythe receiving unit; and the second information processing apparatusincluding: (a) a second receiving unit for receiving, from the firsttransmitting unit: (i) the specific information; and (ii) the attachmentinformation; (b) a storage unit for storing the received specificinformation and attachment information; (c) a reader for reading thefurther attachment information based on the further specific informationinputted by the attachment information input unit; and (d) a secondtransmitter for transmitting the further attachment information read bythe reader to the first information processing apparatus.
 34. Theinformation processing system of claim 33, wherein: (a) the specificinformation input unit includes an image input unit; and (b) thespecific information includes an image of the physical environment. 35.The information processing system of claim 33, wherein: (a) the specificinformation input unit includes a position detecting unit; and (b) thespecific information includes a position in the physical environment.36. The information processing system of claim 33, wherein: (a) thespecific information input unit includes an object identifying unit; and(b) the specific information includes an object in the physicalenvironment.
 37. The information processing system of claim 33, whichincludes an input unit configured to enable a user to input a searchcondition, said further attachment information being selected based onsaid search condition.
 38. The information processing system of claim33, wherein said further attachment information is selected based on aposition in the physical environment.
 39. The information processingsystem of claim 33, wherein the attachment information includes at leastone of voice information, image information, and text information. 40.The information processing system of claim 33, wherein: (a) thesituation in the physical environment is associated with an object inthe physical environment; (b) the specific information includes an IDassociated with the object; and (c) the further attachment informationincludes information which: (i) is related to the object; and (ii) canbe browsed and exchanged.
 41. The information processing system of claim33, wherein: (a) the situation in the physical environment is associatedwith a position in the physical environment; (b) the specificinformation includes position information of the physical environment;and (c) the received further attachment information includes informationwhich: (i) is related to the position; and (ii) can be browsed andexchanged.
 42. An information processing apparatus comprising: aspecific information input unit for inputting specific information tospecify a situation in the real world; an attachment information inputunit for inputting attachment information to be attached to thesituation in the real world specified by the specific information; atransmitting unit for transmitting the attachment information and thespecific information to another information processing apparatus, theanother information processing apparatus storing the attachedinformation that is associated with the specific information; asearching unit for searching the another information processingapparatus to acquire further attachment information based on furtherspecific information inputted by the attachment information input unit;a receiving unit for receiving the further attachment informationtransmitted from the another information processing apparatus; and adisplay controlling unit for controlling to display the furtherattachment information received by the receiving unit.
 43. Theinformation processing apparatus of claim 42, wherein: (a) the specificinformation input unit includes an image input unit; and (b) thespecific information includes an image of the physical environment. 44.The information processing apparatus of claim 42, wherein: (a) thespecific information input unit includes a position detecting unit; and(b) the specific information includes a position in the physicalenvironment.
 45. The information processing apparatus of claim 42,wherein: (a) the specific information input unit includes an objectidentifying unit; and (b) the specific information includes an object inthe physical environment.
 46. The information processing apparatus ofclaim 42, which includes an input unit configured to enable a user toinput a search condition, said further attachment information beingselected based on said search condition.
 47. The information processingapparatus of 42, wherein said further attachment information is selectedbased on a position in the physical environment.
 48. The informationprocessing apparatus of claim 42, wherein the attachment informationincludes at least one of voice information, image information, and textinformation.
 49. The information processing apparatus of claim 42,wherein: (a) the situation in the physical environment is associatedwith an object in the physical environment; (b) the specific informationincludes an ID associated with the object; and (c) the furtherattachment information includes information which: (i) is related to theobject; and (ii) can be browsed and exchanged.
 50. The informationprocessing apparatus of claim 42, wherein: (a) the situation in thephysical environment is associated with a position in the physicalenvironment; (b) the specific information includes position informationof the physical environment; and (c) the received further attachmentinformation includes information which: (i) is related to the position;and (ii) can be browsed and exchanged.
 51. A method of operating aninformation processing apparatus including instructions, the methodcomprising: (a) inputting specific information to specify a situation inthe real world; (b) inputting attachment information to be attached tothe situation in the real world specified by the specific information;(c) transmitting the attachment information and the specific informationto another information processing apparatus, the another informationprocessing apparatus storing the attached information that is associatedwith the specific information; (d) causing a processor to execute theinstructions to search the another information processing apparatus toacquire further attachment information based on further specificinformation inputted by an attachment information input unit; (e)receiving the further attachment information transmitted from theanother information processing apparatus; and (f) causing a display unitto display the received further attachment information.
 52. The methodof claim 51, wherein: (a) the specific information input unit includesan image input unit; and (b) the specific information includes an imageof the physical environment.
 53. The method of claim 51, wherein: (a)the specific information input unit includes a position detecting unit;and (b) the specific information includes a position in the physicalenvironment.
 54. The method of claim 51, wherein: (a) the specificinformation input unit includes an object identifying unit; and (b) thespecific information includes an object in the physical environment. 55.The method of claim 51, which includes causing the processor to executethe instructions to enable a user to input a search condition, saidfurther attachment information being selected based on said searchcondition.
 56. The method of claim 51, wherein said further attachmentinformation is further selected based on a position in the physicalenvironment.
 57. The method of claim 51, wherein the attachmentinformation includes at least one of voice information, imageinformation, and text information.
 58. The method of claim 51, wherein:(a) the situation in the physical environment is associated with anobject in the physical environment; (b) the specific informationincludes an ID associated with the object; and (c) the furtherattachment information includes information which: (i) is related to theobject in the physical environment; and (ii) can be browsed andexchanged.
 59. The method of claim 51, wherein: (a) the situation in thephysical environment is associated with a position in the physicalenvironment; (b) the specific information includes position informationof the physical environment; and (c) the received further attachmentinformation includes information which: (i) is related to the position;and (ii) can be browsed and exchanged.
 60. A computer-readable mediumstoring instruction structured to cause an information processingapparatus to: (a) input specific information to specify a situation inthe real world; (b) input attachment information to be attached to thesituation in the real world specified by the specific information; (c)transmit the attachment information and the specific information toanother information processing apparatus, the another informationprocessing apparatus storing the attached information that is associatedwith the specific information; (d) search the another informationprocessing apparatus to acquire further attachment information based onfurther specific information which is input; (e) receive the furtherattachment information transmitted from the another informationprocessing apparatus; and (f) display the received further attachmentinformation.
 61. An information processing apparatus comprising: areceiving unit for receiving, from another information processingapparatus: (a) specific information to specify a situation in the realworld; and (b) attachment information attached to the situation in thereal world specified by the specific information; a storage unit forstoring the received specific information and attachment information; areader for reading further attachment information based on furtherspecific information inputted by the another information processingapparatus; and a transmitter for transmitting the further attachmentinformation read by the reader to the another information processingapparatus.
 62. The information processing apparatus of claim 61, whereinthe specific information includes at least one of image information ofthe physical environment, position information of the physicalenvironment, and object information of the physical environment.
 63. Theinformation processing apparatus of claim 61, the second attachmentinformation is selected based on a received search condition.
 64. Theinformation processing apparatus of claim 61, the second attachmentinformation is selected based on a position in the physical environment.65. The information processing apparatus of claim 61, wherein: (a) thesituation in the physical environment is associated with an object inthe physical environment; (b) the specific information includes an IDassociated with the object; and (c) the transmitted further attachmentinformation: (i) is related to the object; and (ii) can be browsed andexchanged.
 66. The information processing apparatus of claim 61,wherein: (a) the situation in the physical environment is associatedwith a position in the physical environment; (b) the specificinformation includes position information of the physical environment;and (c) the transmitted further attachment information: (i) is relatedto the position in the physical environment; and (ii) can be browsed andexchanged.